Track feature detection using machine vision

ABSTRACT

The present disclosure generally relates to automated detection of railroad track features. Images of a railroad track are captured and analyzed to identify track features such as anchors, spikes, rail ties, tie plates, and joints. Various image processing techniques are utilized to accurately distinguish between track features and other objects in the captured images. Track features identified in the images are assigned identifiers and locations and stored in a database so that a status and/or condition of the track features may be monitored for maintenance purposes.

CROSS-REFERENCE TO RELATED APPLICATION

This application claims priority to U.S. Provisional Patent ApplicationNo. 62/475,722 filed on Mar. 23, 2017, the disclosure of which is herebyincorporated by reference in entirety.

BACKGROUND

Railroads are typically constructed to include a pair of elongated,substantially parallel rails held in place using spikes, anchors, railties, and various other hardware (e.g., track features). Over time,normal wear and tear on the railroad may require maintenance and/orreplacement of track features. Traditional methods of maintaining railsand accompanying hardware often include manual inspection of therailroad tracks, which is very labor and time intensive. As such,automation of railroad track inspection is desired.

BRIEF SUMMARY

The present disclosure generally relates to a railroad track featuredetection system. The system couples with an underside surface or anoutside surface of a railroad chassis. The system utilizes a camerapointed at the rails on which the railroad chassis operates to captureimages of the railroad track. A high intensity light is used in tandemwith the camera so as to illuminate the railroad track and trackfeatures during image capture. Illuminating railroad track surfacesprovides sufficient lighting for discerning between track discolorationand deliberate colored markers that are used to identify the location ofvarious track features. Additionally, illuminating track features suchas anchors and spikes casts shadows upon the rail surface. During imageprocessing, these shadows help identify track features in the capturedimages.

A variety of image processing techniques are deployed. For example,color values for each pixel in a captured image may be determined andused to identify an object or an object edge in the image. Trackfeatures identified in captured images are assigned identifiersassociated with a type of track feature as well as location informationsuch as global positioning system (GPS) coordinates. Once input into adatabase, this assigned information may enable efficient tracking andmonitoring of a status and/or condition of track features formaintenance purposes.

BRIEF DESCRIPTION OF THE DRAWINGS

Reference is now made to the following descriptions taken in conjunctionwith the accompanying drawings.

FIG. 1 illustrates an exemplary railroad track feature detection systemaccording to the present disclosure.

FIG. 2 illustrates an exemplary computing system according to thepresent disclosure.

FIG. 3A illustrates an exemplary computing environment according to thepresent disclosure.

FIG. 3B illustrates an exemplary connectivity diagram of the computingenvironment of FIG. 3A.

FIG. 4A illustrates exemplary visual processing of a captured railroadtrack image according to the present disclosure.

FIG. 4B illustrates exemplary visual processing of a captured railroadtrack image according to the present disclosure.

FIG. 4C illustrates exemplary visual processing of a captured railroadtrack image according to the present disclosure.

FIG. 5A illustrates an exemplary captured image and an associatedprocessed image of a first anchor type according to the presentdisclosure.

FIG. 5B illustrates an exemplary captured image and an associatedprocessed image of a second anchor type according to the presentdisclosure.

FIG. 5C illustrates an exemplary captured image and an associatedprocessed image of a third anchor type according to the presentdisclosure.

FIG. 5D illustrates an exemplary captured image and an associatedprocessed image of a fourth anchor type according to the presentdisclosure.

FIG. 5E illustrates an exemplary captured image and an associatedprocessed image of a fifth anchor type according to the presentdisclosure.

DETAILED DESCRIPTION

Various embodiments of a railroad track feature detection system aredescribed according to the present disclosure. It is to be understood,however, that the following explanation is merely exemplary indescribing the devices and methods of the present disclosure.Accordingly, several modifications, changes, and substitutions arecontemplated.

As shown in FIG. 1, a railroad track feature detection system 100 mayinclude a railroad chassis 102. In some embodiments, the railroadchassis 102 may be associated with an unmanned rail maintenance vehicle(e.g., a drone vehicle), a manned rail maintenance vehicle, and/oranother rail vehicle. The railroad chassis 102 may include wheels 104for interfacing with underlying railroad tracks (e.g., rails 106). Therailroad chassis 102 may be towed behind another rail vehicle as therail vehicle propels itself along the rails 106 of the railroad track.Alternatively, the railroad chassis 102 may be self propelled and thusmay include an engine, a propulsion system, and/or another operatingsystem (not pictured) for propelling the railroad chassis 102 along therails 106 of the railroad track. In further embodiments, the railroadchassis 102 may take the form of a road-rail chassis or by-rail vehicle,which may be operated on both roads and rail.

As described throughout, the railroad track may include a pair ofelongated, substantially parallel rails 106, which may be coupled to aplurality of laterally extending rail ties 108. In some embodiments, atop surface of each rail tie 108 may be coupled to a bottom surface ofthe rails 106. The rail ties 108 may be disposed on a ballast bed ofhard particulate material such as gravel (e.g., ballast, rocks, and/orthe like) and may be used to support the rails 106.

The railroad track may further include a variety of track features 110used for securing the rails 106 to rail ties 108, the ground, and/orother structures. For example, track features 110 such as spikes oranchors may be used to couple a rail 106 to an underlying rail tie 108.As described herein, a track feature 110 may refer to any type ofrailroad track hardware such as an anchor, a spike, a rail tie, a tieplate, a tie hole, a rail joint, a switch, a rail hole, and/or the like.

Additionally, the rails 106 may include a variety of visual markers 112that are disposed on a surface of the rail 106. These markers 112 may bedrawn or printed onto a surface of the rail 106 (e.g., an insidesurface, an outer surface, a top surface, and/or the like) using paint,an adhesive, lasers, and/or other means. The markers 112 may be utilizedto identify the locations of track features 110 along the rails 106. Forexample, a marker 112 may be disposed on an inside surface of a rail 106at a location immediately adjacent to a track feature 110 (e.g., a railtie or an anchor). Different color markers 112 may be used to denotedifferent types of track features 110 and/or different sections ofrailroad tracks. For example, blue paint markers may be used to identifyanchor locations, whereas green markers may be used to identify railjoint locations. In some embodiments, the markers 112 may even bepainted or printed directly onto track features 110.

The railroad track detection system 100 may further include an enclosure114 for housing various electronic components. For example, theenclosure 114 may house a circuit board or motherboard for executingvarious operations described herein. In some embodiments, the contentsof the enclosure 114 may be pressurized so that dust, vegetation, andother unwanted particulate may not enter the enclosure. As such, alow-pressure compressor 116 may be operatively coupled to the enclosure114. The compressor 116 and/or the enclosure 114 may include one or morepneumatic valves for ensuring a proper seal for maintaining a desiredpressurization of the enclosure 114.

The enclosure 114 may be coupled to the railroad chassis 102 via amounting bracket 118. The mounting bracket 118 may enable removablecoupling of the enclosure 114 and the railroad chassis 102 so that thepositioning of the enclosure 114 relative to the railroad chassis 102 isconfigurable. In some embodiments, the enclosure 114 may be coupled toan underside of the railroad chassis 102. Alternatively, the enclosure114 may be coupled to a side surface of the railroad chassis 102.

The track feature detection system 100 may also include one or morecameras 120. As seen in FIG. 1, a camera 120 may be operatively coupledto the enclosure 114 via a mounting bracket 122. In some embodiments,the mounting bracket 122 may be pivotable so that the camera 120 may bepositioned at an angle relative to the ground. The mounting bracket 122may also be telescopic, mechanized, and/or include one or more jointsfor precisely positioning the camera 120.

The camera 120 may be utilized to capture images of the rails 106 as therailroad chassis 102 is propelled along the rails 106. For example, thecamera 120 may be positioned in such a way as to capture images of aninside surface of the rail 106. In some embodiments, the camera 120 maybe configured to capture a video feed and/or still image frames of therails 106. The camera 120 may include visual sensors, a high-speedcamera, a color camera, a global shutter camera, and/or the like. Thecamera 120 may be enabled to capture high-resolution images ofapproximately 640 pixels by 480 pixels or higher and at high speeds(e.g., approximately 12 frames per second to approximately 30 frames persecond or higher). Additionally, the camera 120 and/or various sensorsmay also measure a height, a width, an orientation, a shape, a contour,an angle, a condition, and/or other factors associated with the rails106, track features 110, and/or color markers 112.

To assist the camera 120 with capturing high-quality images, a light 124may be utilized by the track feature detection system 100. The light 124may be operatively coupled to the enclosure 114 and/or the camera 120via a mounting bracket 122. In some embodiments, the light 124 may bepositioned adjacent to the camera 120. For example, the light 124 may bepositioned such that a longitudinal axis of light being projected fromthe light 124 may be substantially parallel (e.g., on a common plane) asan axis associated with a lens of the camera 120. In other embodiments,the light 124 may be positioned opposite the camera 120 and/or in such away that a longitudinal axis of light being projected from the light 124is angled relative to an axis associated with a lens of the camera 120.

The light 124 may include a high-intensity light such as a lightemitting diode (LED) direct light, a laser, and/or the like. In someembodiments, the light 124 may include multiple lights positioned indifferent positions around the enclosure 114 and/or camera 120.

As the camera 120 captures images of the rail 106, the light 124 may beused to illuminate various track features 110 and/or markers 112 on therails 106. For example, light emitted by the light 124 may cause trackfeatures 110 such as anchors or spikes to cast shadows on the surface ofthe rail 106. These shadows may be detected in captured images toidentify track features 110 at various locations of the rails 106. Forexample, a track feature 110 that produces one shadow of a particularlength at a location on the rail may be identified in a captured imageas a spike, whereas a track feature 110 that produces four shadows of aparticular length and at a different location on the rail may beidentified in a captured image as an anchor.

Additionally, color markers 112 may be illuminated by the light 124 sothat a color or design of each marker 112 may be accurately identifiedin captured rail images. The light 124 may also be helpful indifferentiating between track features 110, color markers 112, shadows,vegetation, and/or stains (e.g., oils, hydraulic fluids, grease, and/orthe like) in captured images. The camera 120 and/or the light 124 may becalibrated so as to set a frame of capture that enables accurateidentification of objects (e.g., track features 110, color markers 112,shadows, vegetation, stains, and/or the like) in captured images. Forexample, an image frame edge and/or contour of the camera 120 may beoffset and/or shifted so that clear images of the rail 106, trackfeatures 110, and/or markers 112 may be captured. In some embodiments, abottom edge of the camera's 120 range of detection may be aligned with afoot of the rail 106. As another example, the intensity and/or directionof the light 124 may be modified so as to reduce unwanted glare and/oradjust for various weather conditions and/or amounts of dust,particulate, and/or vegetation.

Once captured by the camera 120, images of the rails 106 may be sentfrom the camera 120 to the enclosure 114 for processing and/or storage.Various computer hardware described herein may be included in theenclosure 114. Additionally and/or alternatively, captured images may besent from the camera 120 and/or the enclosure to another computingdevice for processing and/or storage. For example, images captured bythe camera 120 may be transmitted by a communication device included inthe enclosure 114 to an off-site and/or cloud-based computing system forprocessing and/or storage. In some embodiments, captured images may betransmitted by components included in the enclosure 114 as they arereceived from the camera 120 (e.g., in real time). Alternatively,captured images may be transmitted at predetermined intervals, when therailroad chassis 102 is determined to be within proximity to aparticular communication network, and/or at another time.

Location information associated with each captured image may also becollected by components included in the enclosure 114. This locationinformation may be used to identify a location on the rails 106 at whicha track feature 110, a color marker 112, and/or another object isdetected. Location information may include GPS coordinates, X-Ycoordinates, and/or other information and may be transmitted with thecaptured images for processing and/or analysis.

Captured images may be analyzed by computing components as describedherein for the purpose of identifying track features 110 in the images.Identified track features 110 may be tagged with identifiers andcorresponding location information in a database. In this manner, thelocations of various track features 110 on the rails 106 may be trackedand monitored for maintenance purposes.

Various computing techniques may be deployed when performing imageprocessing on captured images. For example, techniques such as pixelcolor identification and analysis, support vector machine, edgedetection, histograms of gradients, background subtraction,hue/saturation/value (HSV) values, red-green-blue (RGB) color values,pattern detection, and/or other image processing techniques may be usedby one or more computing processors as described herein. Scales, anglesof capture, length of shutter, shutter speed, brightness, darkness,contrast, hue, saturation, and/or a variety of other characteristics maybe set (e.g., selected and/or weighted) by a user or automatically bythe railroad track feature detection system 100 to ensure accurate andefficient capture and processing of images.

The railroad track feature detection system 100 may identify patterns oftrack features 110 and/or associated shadows, markers 112 as well aslocation information of the same to determine which identifier is to beassigned to an identified object in a captured image. Additionally, therailroad track feature detection system 100 may determine a railroadtrack layout of the rails 106 based on identified track features 110.

Ultimately, the track feature detection system 100 may enable moreefficient documentation and tracking of existing rails 106 and trackfeatures 110. Maps and models of the rails 106 may be generated by thetrack feature detection system 100, where track features 110 and/orcolor markers 112 are displayed digitally. Recommendations forperforming maintenance on various track features 110 may be generatedwithout requiring human inspection. For example, identifying a marker ofa particular color that is associated with a particular rail tie maycause the track feature detection system 100 to recommend a maintenancevehicle to pull (e.g., replace) that particular rail tie. As such,maintenance of the rails 106 and associated track features 110 may bestreamlined in an automated and cost-effective manner.

FIG. 2 illustrates a computing system 200 for enabling the railroadtrack feature detection system 100 of FIG. 1 and/or other railroad trackfeature detection systems described herein. In some embodiments, thecomputing system 200 may include a first server 202 and a second server204.

The first server 202 and the second server 204 may be communicativelycoupled to one another by a network 206 as described herein. As such, avariety of data may be transmitted between the first server 202 and thesecond server 204 during image capture and analysis processes describedherein. The network 206 may include any wireless and/or wiredcommunications network that facilitates communication between the firstserver 202 and the second server 204, as well as between any othercomputing devices (e.g., a user's smartphone, a third party server,and/or the like). For example, the network 206 may include an Ethernetnetwork, a cellular network, a computer network, the Internet, awireless fidelity (Wi-Fi) network, a light fidelity (Li-Fi) network, aBluetooth network, a radio frequency identification (RFID) network, anear-field communication (NFC) network, a laser-based network, and/orthe like. In some embodiments, the network 206 may also include aplurality of networks.

The first server 202 may be associated with the railcar chassis 102 ofFIG. 1 and may be housed in the enclosure 114. For example, the firstserver 202 may include a computer onboard the railroad chassis 102 thatenables operation of electronics included in the enclosure 114, thecompressor 116, the mounting bracket 118, the camera 120, the mountingbrackets 122, and/or the light 124 of FIG. 1. In some embodiments, thefirst server 202 may include a computing device such as a mainframeserver, a content server, a communication server, a laptop computer, adesktop computer, a handheld computing device, a smart phone, a smartwatch, a wearable device, a touch screen, a biometric device, a videoprocessing device, an audio processing device, a cloud-based computingsolution and/or service, and/or the like. The first server 202 may alsoinclude multiple servers configured to communicate with one anotherand/or implement load-balancing techniques described herein.

The first server 202 may include various elements of a computingenvironment as described herein (e.g., computing environment 300 ofFIGS. 3A and 3B). For example, the first server 202 may include aprocessing unit 208, a memory unit 210, an input/output (I/O) unit 212,and/or a communication unit 214. Each of the processing unit 208, thememory unit 210, the input/output (I/O) unit 212, and/or thecommunication unit 214 may include one or more subunits and/or othercomputing instances as described herein for performing operationsassociated with track feature detection.

The second server 204 may be associated with an off-site computingdevice for processing captured images of railroad tracks. For example,the second server 204 may include a cloud-based server that is notonboard the railroad chassis 102 of FIG. 1. In some embodiments, thesecond server 204 may include a computing device such as a mainframeserver, a content server, a communication server, a laptop computer, adesktop computer, a handheld computing device, a smart phone, a smartwatch, a wearable device, a touch screen, a biometric device, a videoprocessing device, an audio processing device, a cloud-based computingsolution and/or service, and/or the like. The second server 204 may alsoinclude multiple servers configured to communicate with one anotherand/or implement load-balancing techniques described herein.

The second server 204 may include various elements of a computingenvironment as described herein (e.g., computing environment 300 of FIG.3A and FIG. 3B). For example, the second server 204 may include aprocessing unit 216, a memory unit 218, an input/output (I/O) unit 220,and/or a communication unit 222. Each of the processing unit 216, thememory unit 218, the input/output (I/O) unit 220, and/or thecommunication unit 222 may include one or more subunits and/or othercomputing instances as described herein for performing operationsassociated with track feature detection.

FIG. 3A and FIG. 3B illustrate exemplary functional and system diagramsof a computing environment 300 for enabling railroad track featuredetection and associated image processing techniques described herein.Specifically, FIG. 3A provides a functional block diagram of thecomputing environment 300, whereas FIG. 3B provides a detailed systemdiagram of the computing environment 300 Additionally, any units and/orsubunits described herein with reference to the computing environment300 of FIG. 3A and/or FIG. 3B may be included in one or more elements ofFIG. 2, such as the first server 202 and/or the second server 204. Thecomputing environment 300 and/or any of its units and/or subunitsdescribed herein may include general hardware, specifically-purposedhardware, and/or software.

The computing environment 300 may include, among other elements, aprocessing unit 302, a memory unit 304, an input/output (I/O) unit 306,and/or a communication unit 308. As described herein, each of theprocessing unit 302, the memory unit 304, the I/O unit 306, and/or thecommunication unit 308 may include and/or refer to a plurality ofrespective units, subunits, and/or elements. Furthermore, each of theprocessing unit 302, the memory unit 304, the I/O unit 306, and/or thecommunication unit 308 may be operatively and/or otherwisecommunicatively coupled with each other so as to facilitate railroadtrack feature detection and associated image analysis techniquesdescribed herein. Further, the processing unit 302, the memory unit 304,the I/O unit 306, and/or the communication unit 308 may refer to theprocessing unit 208, the memory unit 210, the I/O unit 212, and/or thecommunication unit 214 of the first server 202 of FIG. 2. Additionally,the processing unit 302, the memory unit 304, the I/O unit 306, and/orthe communication unit 308 may refer to the processing unit 216, thememory unit 218, the I/O unit 220, and/or the communication unit 222 ofthe second server 202 of FIG. 2.

The processing unit 302 may control any of the one or more units 304,306, 308, as well as any included subunits, elements, components,devices, and/or functions performed by the units 304, 306, 308 includedin the computing environment 300. The described sub-elements of thecomputing environment 300 may also be included in similar fashion in anyof the other units and/or devices included in the system 200 of FIG. 2.Additionally, any actions described herein as being performed by aprocessor may be taken by the processing unit 302 alone and/or by theprocessing unit 228 in conjunction with one or more additionalprocessors, units, subunits, elements, components, devices, and/or thelike. Additionally, while only one processing unit 302 may be shown inFIG. 3A and/or FIG. 3B, multiple processing units may be present and/orotherwise included in the computing environment 300 or elsewhere in theoverall system (e.g., railroad track feature detection system 200 ofFIG. 2). Thus, while instructions may be described as being executed bythe processing unit 302 (and/or various subunits of the processing unit302), the instructions may be executed simultaneously, serially, and/orotherwise by one or multiple processing units.

In some embodiments, the processing unit 302 may be implemented as oneor more computer processing unit (CPU) chips and/or graphical processingunit (GPU) chips and may include a hardware device capable of executingcomputer instructions. The processing unit 302 may execute instructions,codes, computer programs, and/or scripts. The instructions, codes,computer programs, and/or scripts may be received from and/or stored inthe memory unit 304, the I/O unit 306, the communication unit 308,subunits and/or elements of the aforementioned units, other devicesand/or computing environments, and/or the like.

In some embodiments, the processing unit 302 may include, among otherelements, subunits such as a profile management unit 310, a contentmanagement unit 312, a location determination unit 314, a graphicalprocessing unit (GPU) 316, a visual recognition unit 318, a tagging unit320, a tracking unit 322, and/or a resource allocation unit 324. Each ofthe aforementioned subunits of the processing unit 302 may becommunicatively and/or otherwise operably coupled with each other.

The profile management unit 310 may facilitate generation, modification,analysis, transmission, and/or presentation of a profile associated witha railroad track. For example, the profile management unit 310 mayoperate a database associated with a railroad track and may manageinformation associated with the railroad track and/or a railroad chassisas described herein. The profile management unit 310 may also controland/or utilize an element of the I/O unit 232 to enable the captureand/or storage of images of a railroad track. The profile managementunit 310 may receive, process, analyze, organize, and/or otherwisetransform any data received from the user and/or another computingelement so as to generate a profile of a railroad track that includesroute information, status information, track feature information,maintenance information, marker information (e.g., color values), and/orlocation information of the same.

The content management unit 312 may facilitate generation, modification,analysis, transmission, and/or presentation of user interfaces forcontrolling the railroad track feature detection system describedherein. For example, the content management unit 312 may control theaudio-visual environment and/or appearance of application data duringexecution of various processes. Media content for which the contentmanagement unit 312 may be responsible may include advertisements,images, text, themes, audio files, video files, documents, and/or thelike. In some embodiments, the content management unit 312 may alsointerface with a third-party content server and/or memory location.

The location determination unit 314 may facilitate detection,generation, modification, analysis, transmission, and/or presentation oflocation information. Location information may include globalpositioning system (GPS) coordinates, a mile marker, an Internetprotocol (IP) address, a media access control (MAC) address, geolocationinformation, an address, a port number, a zip code, a server number, aproxy name and/or number, device information (e.g., a serial number),and/or the like. In some embodiments, the location determination unit314 may include various sensors, a radar, and/or otherspecifically-purposed hardware elements for enabling the locationdetermination unit 314 to acquire, measure, and/or otherwise transformlocation information. In some embodiments, location-acquiring hardwaremay be included in the enclosure 114 of the system 100 of FIG. 1 fordetermining locations of the railroad chassis 102, track features 110,and/or color markers 114 during operation.

The GPU unit 316 may facilitate generation, modification, analysis,processing, transmission, and/or presentation of images of railroadtracks captured by the camera 120 of FIG. 1. In some embodiments, theGPU unit 316 may be utilized to render visual content for presentationon a user device, analyze a captured image, and/or the like. The GPUunit 316 may also include multiple GPUs and therefore may be configuredto perform and/or execute multiple processes in parallel.

The visual recognition unit 318 may facilitate analysis and/orprocessing of captured images for the purpose of identifying objectsassociated with a railroad track. For example, the visual recognitionunit 318 may be utilized for identifying track features and/or markersin captured images. In some embodiments, the visual recognition unit 318may include GPUs and/or other processing elements so as to enableefficient analysis of images in either series or parallel. The visualrecognition unit 318 may utilize a variety of image analysis techniquessuch as pixel value determination, pixel value comparison, edgedetection, support vector machine, histogram of gradients, backgroundsubtraction, HSV, object recognition, pattern detection, shadowrecognition, image filtering, video and/or image frame sampling, imagesplicing, image reconstruction, error detection, video reconstruction,and/or the like to accurately identify objects of interest in capturedrail images. The visual recognition unit 318 may determine a confidencescore of how likely an object identified in an image is indeed a trackfeature or color marker. For example, the determined confidence scoreassociated with an identified object may be required to exceed athreshold in order for an identifier and/or location information may beassigned to the identified object. The visual recognition unit 318 mayalso be configured to filter out unwanted objects in a captured image.For example, an oil stain on a rail or vegetation that has beenidentified in a captured image may be filtered out using one or moreimage processing techniques. As another example, an object identified inan image that is smaller than a predetermined size (e.g., a number ofpixels) may be filtered out of the image and/or otherwise ignored by thevisual recognition unit 318.

The tagging unit 320 may facilitate the accurate assignment of objectidentifiers and/or location information to identified objects incaptured rail images. Various identifiers such as metadata, colorvalues, numerical values, and/or other tags may be assigned toidentified objects in a database.

The tracking unit 322 may facilitate the monitoring, analysis, and/orprocessing of identifiers and/or location information associated withidentified objects. The tracking unit 322 may monitor a status of anidentified track feature at a particular location in such a manner thatmaintenance for that track feature can be recommended. For example,based on when an image of the track feature was last captured and/or acondition of the track feature detected in the image, the tracking unit322 may generate an alert or recommendation for manually inspecting thetrack feature. Alternatively, the tracking unit 322 may generateinstructions for a drone rail maintenance vehicle to repair and/orinspect the track feature.

The resource allocation unit 324 may facilitate the determination,monitoring, analysis, and/or allocation of computing resourcesthroughout the railroad track feature detection system described herein.For example, the railroad track feature detection system may facilitatea high volume of (e.g., multiple) communication connections between alarge number of supported railroad chassis and/or associated processingservers (e.g., the first server 202 and/or the second server 204 of FIG.2). As such, computing resources of the computing environment 300(and/or any subunit of the aforementioned units) such as processingpower, data storage space, network bandwidth, and/or the like may be inhigh demand at various times during operation. Accordingly, the resourceallocation unit 324 may be configured to manage the allocation ofvarious computing resources as they are required by particular unitsand/or subunits of the computing environment 300 and/or other computingenvironments. In some embodiments, the resource allocation unit 324 mayinclude sensors and/or other specially-purposed hardware for monitoringperformance of each unit and/or subunit of the computing environment300, as well as hardware for responding to the computing resource needsof each unit and/or subunit. In some embodiments, the resourceallocation unit 324 may utilize computing resources of a secondcomputing environment separate and distinct from the computingenvironment 300 to facilitate a desired operation.

For example, the resource allocation unit 324 may determine a number ofsimultaneous communication connections and/or incoming requests for dataand/or image processing. The resource allocation unit 324 may thendetermine that the number of simultaneous communication connectionsand/or incoming requests for meets and/or exceeds a predeterminedthreshold value. Based on this determination, the resource allocationunit 324 may determine an amount of additional computing resources(e.g., processing power, storage space of a particular non-transitorycomputer-readable memory medium, network bandwidth, and/or the like)required by the processing unit 302, the memory unit 304, the I/O unit306, the communication unit 308, and/or any subunit of theaforementioned units for enabling safe and efficient operation of thecomputing environment 300 while supporting the number of simultaneouscommunication connections and/or incoming requests. The resourceallocation unit 324 may then retrieve, transmit, control, allocate,and/or otherwise distribute determined amount(s) of computing resourcesto each element (e.g., unit and/or subunit) of the computing environment300 and/or another computing environment.

In some embodiments, factors affecting the allocation of computingresources by the resource allocation unit 324 may include the number ofongoing communication connections and/or other communication channelconnections, a number of image analysis and/or reporting processes, aduration of time during which computing resources are required by one ormore elements of the computing environment 300, and/or the like. In someembodiments, computing resources may be allocated to and/or distributedamongst a plurality of second computing environments included in thecomputing environment 300 based on one or more factors mentioned above.In some embodiments, the allocation of computing resources of theresource allocation unit 324 may include the resource allocation unit324 flipping a switch, adjusting processing power, adjusting memorysize, partitioning a memory element, transmitting data, controlling oneor more input and/or output devices, modifying various communicationprotocols, and/or the like. In some embodiments, the resource allocationunit 324 may facilitate utilization of parallel processing techniquessuch as dedicating a plurality of GPUs included in the processing unit302 for processing a high-quality video stream of a captured railimages.

In some embodiments, the memory unit 304 may be utilized for storing,recalling, receiving, transmitting, and/or accessing various filesand/or information during operation of the computing environment 300.For example, the memory unit 304 may be utilized for storing imageframes, video streams, storing, recalling, and/or updating railroadtrack profile information, marker information, track featureinformation, and/or the like. The memory unit 304 may include varioustypes of data storage media such as solid state storage media, hard diskstorage media, and/or the like. The memory unit 304 may includededicated hardware elements such as hard drives and/or servers, as wellas software elements such as cloud-based storage drives. For example,the memory unit 304 may include various subunits such as an operatingsystem unit 326, an application data unit 328, an applicationprogramming interface (API) unit 330, a profile storage unit 332, acontent storage unit 334, an object storage unit 336, a secure enclave338, and/or a cache storage unit 340.

The memory unit 304 and/or any of its subunits described herein mayinclude random access memory (RAM), read only memory (ROM), and/orvarious forms of secondary storage. RAM may be used to store volatiledata and/or to store instructions that may be executed by the processingunit 302. For example, the data stored may be a command, a currentoperating state of the computing environment 300, an intended operatingstate of the computing environment 300, and/or the like. As a furtherexample, data stored in the memory unit 304 may include instructionsrelated to various methods and/or functionalities described herein. ROMmay be a non-volatile memory device that may have a smaller memorycapacity than the memory capacity of a secondary storage. ROM may beused to store instructions and/or data that may be read during executionof computer instructions. In some embodiments, access to both RAM andROM may be faster than access to secondary storage. Secondary storagemay be comprised of one or more disk drives and/or tape drives and maybe used for non-volatile storage of data or as an over-flow data storagedevice if RAM is not large enough to hold all working data. Secondarystorage may be used to store programs that may be loaded into RAM whensuch programs are selected for execution. In some embodiments, thememory unit 304 may include one or more databases for storing any datadescribed herein. Additionally or alternatively, one or more secondarydatabases located remotely from the computing environment 300 may beutilized and/or accessed by the memory unit 304.

The operating system unit 326 may facilitate deployment, storage,access, execution, and/or utilization of an operating system utilized bythe computing environment 300 and/or any other computing environmentdescribed herein. In some embodiments, the operating system may includevarious hardware and/or software elements that serve as a structuralframework for enabling the processing unit 302 to execute variousrailroad track feature detection and/or image processing operationsdescribed herein. The operating system unit 326 may further storevarious pieces of information and/or data associated with operation ofthe operating system and/or the computing environment 300 as a whole,such as a status of computing resources (e.g., processing power, memoryavailability, resource utilization, and/or the like), runtimeinformation, modules to direct execution of operations described herein,and/or the like.

The application data unit 328 may facilitate deployment, storage,access, execution, and/or utilization of an application utilized by thecomputing environment 300 and/or any other computing environmentdescribed herein (e.g., the first server 202 and/or the second server204 of FIG. 2). For example, users may be required to download, access,and/or otherwise utilize a software application on a computing device inorder for various operations described herein to be performed. As such,the application data unit 328 may store any information and/or dataassociated with the application. Information included in the applicationdata unit 328 may enable a user and/or computer processor to executevarious operations described herein. The application data unit 328 mayfurther store various pieces of information and/or data associated withoperation of the application and/or the computing environment 300 as awhole, such as a status of computing resources (e.g., processing power,memory availability, resource utilization, and/or the like), runtimeinformation, modules to direct execution of operations described herein,and/or the like.

The API unit 300 may facilitate deployment, storage, access, execution,and/or utilization of information associated with APIs of the computingenvironment 300 and/or any other computing environment described herein.For example, the computing environment 300 may include one or more APIsfor enabling various devices, applications, and/or computingenvironments to communicate with each other and/or utilize the samedata. Accordingly, the API unit 330 may include API databases containinginformation that may be accessed and/or utilized by applications and/oroperating systems of other devices and/or computing environments. Insome embodiments, each API database may be associated with a customizedphysical circuit included in the memory unit 304 and/or the API unit330. Additionally, each API database may be public and/or private, andso authentication credentials may be required to access information inan API database.

The profile storage unit 332 may facilitate deployment, storage, access,and/or utilization of information associated with profiles of railroadtracks, track features, and/or markers. For example, the profile storageunit 332 may store a track profile, identification information, markerinformation, track feature information, location information, and/ormetadata associated with a railroad track and/or a railroad chassis. Insome embodiments, the profile storage unit 332 may communicate with theprofile management unit 310 to receive and/or transmit informationassociated with a railroad profile.

The content storage unit 334 may facilitate deployment, storage, access,and/or utilization of information associated with requested content bythe computing environment 300 and/or any other computing environmentdescribed herein. For example, the content storage unit 334 may storeone or more user interfaces, application information, captured images ofrails, video feeds of rails, processed image frames, audio content,and/or metadata to be presented to a user and/or otherwise utilizedduring operations described herein. In some embodiments, the contentstorage unit 334 may communicate with the content management unit 312 toreceive and/or transmit content files.

The object storage unit 336 may facilitate deployment, storage, access,analysis, and/or utilization of information associated with objectsidentified in image. For example, the object storage unit 336 may storelocation information and/or identifiers associated with objectsidentified in captured image frames. In some embodiments, the objectstorage unit 336 may communicate with the GPUs 316, the visualrecognition unit 318, the tagging unit 320, and/or the tracking unit 322to facilitate analysis of any stored image frame and/or associatedinformation.

The secure enclave 338 may facilitate secure storage of data. In someembodiments, the secure enclave 338 may include a partitioned portion ofstorage media included in the memory unit 304 that is protected byvarious security measures. For example, the secure enclave 338 may behardware secured. In other embodiments, the secure enclave 338 mayinclude one or more firewalls, encryption mechanisms, and/or othersecurity-based protocols. Authentication credentials of a user may berequired prior to providing the user access to data stored within thesecure enclave 338.

The cache storage unit 340 may facilitate short-term deployment,storage, access, analysis, and/or utilization of data. In someembodiments, the cache storage unit 340 may serve as a short-termstorage location for data so that the data stored in the cache storageunit 340 may be accessed quickly. In some embodiments, the cache storageunit 340 may include RAM and/or other storage media types that enablequick recall of stored data. The cache storage unit 340 may include apartitioned portion of storage media included in the memory unit 304.

The I/O unit 306 may include hardware and/or software elements forenabling the computing environment 300 to receive, transmit, and/orpresent information. For example, elements of the I/O unit 306 may beused to capture images of railroad tracks, display images of railroadtracks, receive user input from a user via a user device, presentinformation to a user, and/or the like. In this manner, the I/O unit 306may enable the computing environment 300 to interface with a railroadtrack and/or a human user. As described herein, the I/O unit 306 mayinclude subunits such as an I/O device 342, an I/O calibration unit 344,and/or video driver 346.

The I/O device 342 may facilitate the receipt, transmission, processing,presentation, display, input, and/or output of information as a resultof executed processes described herein. In some embodiments, the I/Odevice 342 may include a plurality of I/O devices. For example, the I/Odevice 342 may include a variety of elements that enable capturing ofimages of railroad tracks such as a camera, a sensor, a light, and/orthe like. The I/O device 342 may also include hardware for interfacingwith a user, such as a keyboard, a touchscreen, a button, a sensor, abiometric scanner, a laser, a microphone, a camera, and/or anotherelement for receiving and/or collecting input from a user. Additionallyand/or alternatively, the I/O device 342 may include a display, ascreen, a sensor, a vibration mechanism, a light emitting diode (LED), aspeaker, a radio frequency identification (RFID) scanner, and/or anotherelement for presenting and/or otherwise outputting data to a user. Insome embodiments, the I/O device 342 may communicate with one or moreelements of the processing unit 302 and/or the memory unit 304 toexecute operations described herein.

The I/O calibration unit 344 may facilitate the calibration of the I/Odevice 342. For example, the I/O calibration unit 344 may detect and/ordetermine one or more settings of the I/O device 342, and then adjustand/or modify settings so that the I/O device 342 may operate moreefficiently.

In some embodiments, the I/O calibration unit 344 may utilize a driver346 (or multiple drivers) to calibrate the I/O device 342. For example,a video driver 346 may be installed on a computer that enables a camerato capture images of railroad tracks in a particular manner. In someembodiments, the I/O device 342 may be calibrated by the I/O calibrationunit 344 based on information included in the driver 346.

The communication unit 308 may facilitate establishment, maintenance,monitoring, and/or termination of communications (e.g., a communicationconnection) between computing devices of the railroad track featuredetection system described herein. The communication unit 308 mayfurther enable communication between various elements (e.g., unitsand/or subunits) of the computing environment 300. In some embodiments,the communication unit 308 may include a network protocol unit 348, anAPI gateway 350, an encryption engine 352, and/or a communication device354. The communication unit 308 may include hardware and/or softwareelements.

The network protocol unit 348 may facilitate establishment, maintenance,and/or termination of a communication connection between computingenvironment 300 and another computing environment (e.g., the firstserver 202 and the second server 204 of FIG. 2) by way of a network. Forexample, the network protocol unit 348 may detect and/or define acommunication protocol required by a particular network and/or networktype. Communication protocols utilized by the network protocol unit 348may include Wi-Fi protocols, Li-Fi protocols, cellular data networkprotocols, Bluetooth® protocols, WiMAX protocols, Ethernet protocols,powerline communication (PLC) protocols, and/or the like. In someembodiments, facilitation of communication between the computingenvironment 300 and any other device, as well as any element internal tothe computing environment 300, may include transforming and/ortranslating data from being compatible with a first communicationprotocol to being compatible with a second communication protocol. Insome embodiments, the network protocol unit 348 may determine and/ormonitor an amount of data traffic to consequently determine whichparticular network protocol is to be used for establishing a videocommunication connection, transmitting data, and/or performing otheroperations described herein.

The API gateway 350 may facilitate the enablement of other devicesand/or computing environments to access the API unit 330 of the memoryunit 304 of the computing environment 300. For example, a user devicemay access the API unit 330 via the API gateway 350. In someembodiments, the API gateway 350 may be required to validate usercredentials associated with a user of a user device prior to providingaccess to the API unit 330 to the user. The API gateway 350 may includeinstructions for enabling the computing environment 300 to communicateand share information with another device.

The encryption engine 352 may facilitate translation, encryption,encoding, decryption, and/or decoding of information received,transmitted, and/or stored by the computing environment 300. Using theencryption engine, each transmission of data may be encrypted, encoded,and/or translated for security reasons, and any received data may beencrypted, encoded, and/or translated prior to its processing and/orstorage. In some embodiments, the encryption engine 352 may generate anencryption key, an encoding key, a translation key, and/or the like,which may be transmitted along with any data content.

The communication device 354 may include a variety of hardware and/orsoftware specifically purposed to enable communication between thecomputing environment 300 and another device, as well as communicationbetween elements of the computing environment 300. In some embodiments,the communication device 354 may include one or more radio transceivers,chips, analog front end (AFE) units, antennas, processing units, memory,other logic, and/or other components to implement communicationprotocols (wired or wireless) and related functionality for facilitatingcommunication between the computing environment 300 and any otherdevice. Additionally and/or alternatively, the communication device 354may include a modem, a modem bank, an Ethernet device such as a routeror switch, a universal serial bus (USB) interface device, a serialinterface, a token ring device, a fiber distributed data interface(FDDI) device, a wireless local area network (WLAN) device and/or devicecomponent, a radio transceiver device such as code division multipleaccess (CDMA) device, a global system for mobile communications (GSM)radio transceiver device, a universal mobile telecommunications system(UMTS) radio transceiver device, a long term evolution (LTE) radiotransceiver device, a worldwide interoperability for microwave access(WiMAX) device, and/or another device used for communication purposes.

FIG. 4A depicts an exemplary visual processing 400 of a capturedrailroad track image. An image 402 of a surface of a rail 404 may becaptured by a camera of the railroad track feature detection system.Water 406 has been poured on the rail 404 to show how the railroad trackfeature detection system described herein may differentiate betweenblemishes and/or stains on the rail 404 from other objects.

The image 402 may be taken by the camera in tandem with a light shiningon the rail surface. By using a light, track features 408 may beilluminated for easy visual detection. Additionally, the light allowsthe track features 408 to cast shadows 410 on the surface of the rail404. The light also illuminates markers 412 painted onto the surface ofthe rail 404 so that its exact color may be easily identified.Additional markers 414 have been superimposed on the image 402 forsimulation.

As seen in processed image 416, with all white, color, and/or lightremoved, a “negative” image is produced. This negative image highlightsthe shadows 410 cast by the track features 408. In this manner, trackfeatures may be identified using shadow detection and/or backgroundsubtraction techniques. Using these techniques, the water stain 406 andmarkers 412, 414 are filtered out of the processed image 416.

In processed image 418, the water 406, track features 408, shadows 410,and marker 412 have been filtered out. The processed image 418 shows howdifferent color markers may be identified by the railroad track featuredetection system. A range of colors can be detected in captured images.Markers of a color that falls within a desired range may appear moresolid, like the rightmost marker 414 in the processed image 418, thanmarkers of a color that fall outside a desired range (e.g., marker 412).In some embodiments, each color may be assigned a range of hue valuesassociated with a color. For example, on a color wheel with 360 degreesof colors, a first color may be assigned to degrees 1-10, whereas asecond color may be assigned to degrees 11-20, and so on. The railroadtrack feature detection system may be configured to search capturedimages for a particular color and/or color range. For example, therailroad track feature detection system may search captured images forcolors associated with a hue within the range of degrees 1-20. In thismanner, HSV processing may identify color markers. The railroad trackfeature detection system may also cycle through multiple hue and/orcolor ranges at a particular location so that markers of multiple colorsmay be identified. As described above, different color markers may beassociated with different track features.

Additionally, shadows may be utilized to identify track features. Forexample, a number of shadows, a size or length of a shadow, a locationof a shadow, and/or other characteristics may be used to identify trackfeatures.

FIG. 4B depicts an exemplary visual processing 420 of a capturedrailroad track image. Similar to above, captured image 422 showssuperimposed markers 426 of various colors. Processed image 424 showshow markers of some colors are filtered out using HSV processing,whereas markers of other colors are easily identified. When searchingfor markers of a particular color, the railroad track feature detectionsystem may search based on hue values only (e.g., saturation, value,brightness, transparency, and/or the like may be set to zero).Alternatively, a variety of values associated with a color may beutilized for searching for markers of particular colors. Using edgedetection (e.g., at locations where neighboring pixels have color and/orhue values that exceed a predetermined threshold), the railroad trackfeature detection system may identify a contour, a moment, and/oroutline of a track feature, a shadow, a marker, and/or another object.

FIG. 4C depicts an exemplary visual processing 426 of a capturedrailroad track image. Similar to above, captured image 428 illustratesan image of a surface of a rail 430 with two track features 432, eachcasting a shadow 434 on the surface of the rail 430. The rail 430 alsoincludes a water stain 436 and a grease stain 438.

Processed image 440 shows how the grease stain 438 may be incorrectlyidentified as a track feature shadow 434 by the railroad track featuredetection system. Error detection processes may be utilized to eliminateerrors. For example, the grease stain 438 may be filtered out from animage frame based on its size. If the grease stain 438 (or anotherobject that should not be identified as a track feature and/or marker)is smaller than a predetermined threshold size, then the railroad trackfeature detection system may filter out the grease stain 438.

It is to be appreciated that the processed images according to thepresent disclosure can identify various anchor types, such as Ericson,stead, unit, woodings and improved fair anchor types. FIG. 5Aillustrates an exemplary captured image 500 and an associated processedimage 502 of a first anchor type 504 according to the presentdisclosure. The boxes in the processed image 502 indicate an identifiedtrack feature (e.g., an anchor). Other track features shown in thecaptured image 500, such as spikes 506, may also be identified in theprocessed image 502.

FIG. 5B illustrates an exemplary captured image 510 and an associatedprocessed image 512 of a second anchor type 514 according to the presentdisclosure. The box in the processed image 512 indicates an identifiedtrack feature (e.g., an anchor). Other track features shown in thecaptured image 510, such as spikes 516, may also be identified in theprocessed image 512.

FIG. 5C illustrates an exemplary captured image 520 and an associatedprocessed image 522 of a third anchor type 524 according to the presentdisclosure. The boxes in the processed image 522 indicate an identifiedtrack feature (e.g., an anchor). Other track features shown in thecaptured image 520, such as spikes 526, may also be identified in theprocessed image 522.

FIG. 5D illustrates an exemplary captured image 530 and an associatedprocessed image 532 of a fourth anchor type 534 according to the presentdisclosure. The red boxes in the processed image 532 indicate anidentified track feature (e.g., an anchor). Other track features shownin the captured image 530, such as spikes 536, may also be identified inthe processed image 532.

FIG. 5E illustrates an exemplary captured image 540 and an associatedprocessed image 542 of two different anchor types 504 and 534 as well aspulled spike 546 according to the present disclosure. The boxes in theprocessed image 542 indicate identified track features (e.g., an anchorand spike).

While various implementations in accordance with the disclosedprinciples have been described above, it should be understood that theyhave been presented by way of example only, and are not limiting. Thus,the breadth and scope of the implementations should not be limited byany of the above-described exemplary implementations, but should bedefined only in accordance with the claims and their equivalents issuingfrom this disclosure. Furthermore, the above advantages and features areprovided in described implementations, but shall not limit theapplication of such issued claims to processes and structuresaccomplishing any or all of the above advantages.

What is claimed is:
 1. A railroad track feature detection systemcomprising: a camera; at least one light source; and a computingapparatus comprising: at least one memory comprising instructions; andat least one processing device configured to execute the instructions,wherein the instructions cause the at least one processing device toperform the operations comprising: capturing, using the camera, an imageof a railroad track, wherein the at least one light source is used tocapture the image, the image being composed of pixels; determining,using a graphical processing unit (GPU) comprised in the at least oneprocessing device, at least one color value of each pixel comprised inthe image; identifying, using a visual recognition unit comprised in theat least one processing device, an object in the image based on thedetermined color values, wherein the at least one processing device isconfigured to measure a shadow to identify a railroad track feature; andassigning, using a tagging unit comprised in the at least one processingdevice, an identifier associated with the railroad track feature and alocation to the identified object in a database.
 2. The railroad trackfeature detection system of claim 1, wherein: the railroad track featureis an anchor or a spike; and the color value comprises a red-green-blue(RGB) value.
 3. The railroad track feature detection system of claim 1,wherein the operations further comprise: assigning a first range of huevalues to correspond to a first color; assigning a second range of huevalues to correspond to a second color; and assigning a third range ofhue values to correspond to a third color.
 4. The railroad track featuredetection system of claim 1, wherein the at least one color valueincludes a hue/saturation/value (HSV) value or a red-green-blue (RGB)value.
 5. The railroad track feature detection system of claim 1,wherein: the object comprises a color marker; and the operations furthercomprise identifying a contour based on the determined color values ofadjacent pixels being greater than a predetermined threshold.
 6. Therailroad track feature detection system of claim 1, wherein the lightsource is operable to cast a shadow of the railroad track feature. 7.The railroad track feature detection system of claim 1, wherein the atleast one processing device is configured to measure a size or a lengthof the shadow and the railroad track feature is identified based on themeasured shadow.
 8. A method for detecting a railroad track feature, themethod comprising: capturing, using a camera and a light source, animage of a railroad track, the image being composed of pixels;determining at least one color value of each pixel comprised in theimage; identifying multiple color ranges, each color range of themultiple color ranges including a corresponding range of color valuesthat correspond to at least one respective color; for at least onepixel, performing a comparison of the at least one color value of thepixel to the color values to at least one of the multiple color ranges;identifying an object in the image based on the determined color valuesand the comparison; measuring a shadow to identify a railroad trackfeature; and assigning an identifier associated with the railroad trackfeature and a location to the identified object.
 9. The method of claim8, further comprising positioning the light source at an angle to thecamera.
 10. The method of claim 9, further comprising using the lightsource to cast the shadow of the railroad track feature and measuring asize or a length of the shadow.
 11. The method of claim 8, furthercomprising determining a confidence score of the object.
 12. The methodof claim 11, further comprising: detecting a color marker on therailroad track; and assigning an identifier and a location to the colormarker.
 13. The method of claim 8, further comprising: assigning a firstrange of color values to correspond to a first color; and comparing theat least one color value of each pixel to the first range of colorvalues.
 14. The method of claim 8, further comprising identifying acontour of the railroad track.
 15. The method of claim 8, furthercomprising filtering unwanted objects in the captured image.
 16. Themethod of claim 8, further comprising determining a hue/saturation/value(HSV) value of each pixel.
 17. A system for detecting features on a railtrack, the system comprising: a camera; at least one light source; aprocessor; and a memory device coupled to the processor, the memorydevice comprising instructions that, when executed by the processor,cause the processor to: illuminate, by the at least one light source, aportion of a railroad track; capture, by the camera, an image of theilluminated portion of the railroad track, the image being composed ofpixels; detect one or more shadow portions in the image, the shadowportions corresponding to a shadow cast by a track feature; measure theshadow portions; and identify the track feature based on the measuredshadow portions.
 18. The system of claim 17, wherein the processor isfurther configured to produce a negative image of the captured image.19. The system of claim 17, wherein the processor is further configuredto: determine at least one color value of each pixel comprised in theimage; identify an object in the image based on the determined colorvalues; and assign an identifier associated with the railroad trackfeature and a location to the identified object in a database.
 20. Thesystem of claim 17, wherein the processor is further configured to:measure a size or a length of the shadow portions; or identify the trackfeature from a plurality of identifiable types of track features.